Final Team ReportThe Great DesignersShawn Drake, Susan Gardner, Mary Norris, Barbara Solomon & Erin Turner Report PDF
Executive Summary
The Great Designers developed a Book Builder (www.bookbuilder.cast.org/) product for high school students which explores the socio-economic impact of the Great Depression,specifically how people relaxed and entertained themselves during these harsh times. Our creative instructional design approach to this project incorporated text, video, photos, and music clips to create a highly interactive and engaging learning experience. Our goal was to create a multimedia product that while specific in its scope (i.e., the impact of the Great Depression on leisure and entertainment), would be engaging and stimulate student interest by appealing to students' need for more interactive content. Universal Design for Learning (UDL) materials were used to provide alternative pathways to success including multiple modes of presentation, varied levels of support and challenge, and options for recruiting and sustaining interest and motivation. Our product was designed to be self-instructional and can be used either in the classroom or as homework. We incorporated the ADDIE framework in our instructional design process. During the analysis stage we completed the following analyses: needs analysis, goal analysis, learner analysis, task analysis, and content analysis. During the formative evaluation process, the lesson was administered to students, peer evaluators, and subject matter experts. Modifications were completed to make the lesson more accessible and engaging to students. The summative evaluation measures include measures of students’ learning as well as measures of student engagement.
Analysis
Objectives
The purpose of the lesson was to raise student scores on the End of Course (EOC) exam in required South Carolina high school U.S. History and Constitution courses by raising student interest and engagement. The Great Designers adopted state curriculum standards for U.S. History and Constitution as the project objectives. The specific standards addressed by the lesson are listed below with the relevant text regarding lesson objectives shown in bold:
USHC-6.1 Explain the impact of the changes in the 1920s on the economy, society, and culture, including the expansion of mass production techniques, the invention of new home appliances, the introduction of the installment plan, the role of transportation in changing urban life, the effect of radio and movies in creating a national mass culture, and the cultural changes exemplified by the Harlem Renaissance.
USHC-6.3 Explain the causes and consequencesof the Great Depression, including the disparities in income and wealth distribution; the collapse of the farm economy and the effects of the Dust Bowl; limited governmental regulation; taxes, investment; and stock market speculation; policies of the federal government and the Federal Reserve System; and the effects of the Depression on the people.
The ultimate goal of our lesson was to generate an increase in student engagement, and thereby an increase in student achievement, on U.S. History and Constitution End of Course (EOC) exam scores. The incorporation of technology in the presentation of material was an over-riding consideration in keeping with the desire to incorporate principles of UDL for effectiveness and accessibility of content information.
Analysis Process
The Great Designers completed a needs analysis, goal analysis, learner analysis, task analysis, and content analysis. Each team member led one analysis with contributions from the other team members. To summarize the results of our analyses, each team member developed a graphic organizer to visually present his or her assigned analysis. A needs analysis was essential to ensure that there was in fact a need for the lesson and to determine the specific focus of the need (i.e., identifying relevant subtopics within the broad topic of the Great Depression). After determining the need, The Great Designers conducted a goal analysis based on the curriculum standards and goals identified in the needs analysis. A learner analysis was essential since most of the team members did not have experience working with the high school population or teaching U.S. History and Constitution. A task analysis identified the general format of and activities to be included in the Book Builder lesson, an important step since the purpose of the lesson was to create a more interactive and engaging learning activity for students. Despite narrowing the broad topic of the Great Depression down to the subtopic of leisure and entertainment, this subtopic was still too broad for the current Book Builder project. A content analysis enabled the team to further narrow the focus to specific types of leisure and entertainment activities and ensure content veracity and usefulness.
Needs Analysis
The initial analysis for topic selection was largely derived from examination of information made available from the South Carolina Department of Education regarding student performance on End of Course (EOC) assessment data (SCDE EOCEP, 2012). U.S. History and Constitution is a required course for high school graduation in the state of South Carolina. In 2005 the SC Department of Education implemented an EOC exam for all students enrolled in U.S. History and Constitution. Since the implementation of the EOC exam, scores have been consistently low with over 71% of students earning a D or F. Additionally, no more than 3% of students have attained a grade of A in the most recent report available in 2011. The EOC exam not only comprises 20% of students' final grades, but also impacts schools' and districts' report cards and state funding. Anecdotal evidence suggests that the low EOC exam performance is due to a mismatch between student needs and teacher instructional delivery methods as well as low student interest in U.S. history in general. Furthermore, students do not see the importance of connecting their historical past to their present and future. The Great Depression is included in the curriculum standards for the U.S. History course and consequently is included on the EOC exam. This evidence supports the general need to retool teaching strategies and learning opportunities in U.S. History and Constitution classes and the era of the Great Depression was chosen due to a great amount of resources that can be successfully incorporated into a product such as Book Builder. Goal Analysis As stated in our initial Goal Analysis on the project wiki, our goal was to, “generate student interest in eleventh grade United States History.” With this in mind objectives were selected that were both supported by and found within S.C. state standards for U.S. History and Constitution as noted above. These objectives were selected to address the cognitive domains found in Bloom’s Taxonomy regarding the lower levels of cognition (identifying), and expanding somewhat to incorporate this information into higher levels for purposes of explanation (Bloom, Engelhart, Furst, Hill, Krathwohl, 1956, p. 95, 2011). Upon completion of the instructional unit measurements may then be made by means of data collected from the post-test to determine the effectiveness of the instructional module, and if necessary, the incorporation of further instruction.
Learner Analysis
The student traditionally enrolled in U.S. History and Constitution is 16 to 17 years old, in the 11th grade, male or female, and from diverse racial and ethnic groups. The ability level of students range from challenged to gifted and talented, but the greater majority fall in the average range, and this is the case with technical proficiency as well. Under normal circumstances an intervention of this type might be targeted towards the lower performers, but given the data that over 70% of all test-takers statewide are at a below average or failing level it was reasonable to assert that the average student still falls within the scope of students requiring an intervention such as The Great Designer’s project. Likewise, the technical ability of students may be affected by the socio-economic conditions found in the school utilizing the intervention which may limit extended knowledge of the integrated technologies within the project. This may also pose limitations insofar as that students utilizing the project may have limitations with access to the internet at home which may, in turn, limit usage for purposes of reinforcement and/or homework beyond the classroom setting.
Task Analysis
The product was created and developed to be self-instructional in nature, and with this in mind efforts were made to make it as user-friendly as possible. Tasks incorporated into the product itself were largely universal features found in traditional web pages such as hyperlinks to supporting materials, videos, audio segments, etc. Where the incorporated tasks were less intuitive, i.e. use of the glossary feature, etc., an explanatory video was included at the outset to explain those features more in-depth so that the user would be able to successfully navigate the product with little difficulty. Aiding these modification decisions were data from formative evaluations which enabled the team to make decisions regarding the content presentation, page formatting, and user enabled features.
Content Analysis
Of all of our analyses, the content analysis probably changed the most throughout the instructional design process. Because the Great Depression is a very broad topic, we found it necessary to narrow the lesson topic and content several times during the design, development, implementation, and evaluation phases. As can be seen in our content analysis presented on our wiki, our original scope of content for inclusion in the project was much broader, including causes of the Great Depression, the New Deal, and general vocabulary like shanty towns, inflation, and recession. After much discussion and trial-and-error, we gradually refined the topic to focus specifically on leisure and entertainment (see the Music/Arts & Culture section of the content analysis). We ensured our content was valid and complete by aligning the content with the aforementioned curriculum standards as well as by conducting formative subject matter expert (SME) evaluations. Based on data from our SME evaluators and formative evaluation surveys, we modified our content by once again narrowing it to directly support the objectives chosen in the goal and content analysis.
Design and Development
Description of the instruction, setting, activities, and sequence
The Book Builder lesson is designed to be implemented in a high school U.S. History and Constitution classroom with access to computers and high-speed internet (see below for details). The following sequence and explanation for instruction is recommended:
Topic introduction - After having covered the standards on the causes of the Great Depression the teacher will introduce this lesson as one of the major consequences of the Great Depression. The teacher will ask the students to write down what they know about entertainment during the Great Depression. They will keep that list until the end of the lesson in order to see how their lists compared to what they learned.
Computer set-up - Students will be assigned to individual computer workstations in their classrooms. If the regular history classrooms do not have computer workstations, computer labs within the high schools can be used. Each workstation must have Internet access, capabilities to play video and audio clips, and headphones to minimize noise disruptions. The teacher should have access to a Smart Board or the ability to project his or her computer screen for the rest of the class to view.
Pre-test - Prior to starting the Book Builder lesson, the students will take an online pre-test to measure their beginning knowledge of the lesson content. This pre-test may be completed either the day before the lesson or the day of the lesson.
Teacher-led overview and whole group instruction - The lesson will begin with the teacher providing the URL to the Book Builder lesson and a general overview of how to use the features within Book Builder, such as the coaches, glossary, table of contents, and student response areas. The teacher should demonstrate these features on a Smart Board or on a projected computer screen. The teacher will also provide instructions on how to complete the lesson and the student response areas. The teacher's expectations of the students will also be discussed. Students will be instructed to visit each multimedia link within the Book Builder lesson. The teacher is strongly encouraged to guide the students through the first 2 or 3 pages of the Book Builder lesson by completing those pages together as a class. This strategy will allow the teacher to be more specific in how to use each feature, how to access the multimedia links, and how to develop an appropriate student response. The teacher can instruct students to answer the student response prompts directly into Book Builder to be printed or saved as a text file at the end of class. Another option is for the teacher to have the students write their responses on a separate sheet of paper to be turned in at the end of class. The student responses can be used as a formative evaluation or can be a graded assignment. If the responses will be graded, the teacher should encourage the students to review the Book Builder lesson outside of class to ensure the responses are completed to a high standard.
Independent practice and teacher monitoring - After the initial instructions and overview, the students will be asked to complete the Book Builder lesson independently. For the remainder of the lesson, the teacher will monitor the students' use of Book Builder, their engagement with the lesson, technical or learning difficulties, responses to learning prompts, and progress toward completion. Although the teacher is not directly delivering the content of the lesson, the teacher should be highly involved with the students throughout the lesson. At the teacher’s discretion, small group and class discussions can be incorporated during this step. Several of the student response prompts lend themselves easily to group discussion.
Post-test - Immediately after completing the Book Builder lesson, the students will take an online post-test to measure their knowledge of the lesson content and the effectiveness of the lesson in meeting the learning objectives. Included in the post-test, students will complete a brief student satisfaction survey to determine their overall satisfaction and engagement with the Book Builder lesson and the use of Book Builder as a learning tool.
Development process supporting the instructional approach
Major components
The major components for instruction in our project begin with 1.) a pretest assessment of current student ability, and possibly to activate prior knowledge with respect to the content. 2.) The introductory presentation will then be followed by the use of the product for support and further enquiry. 3.) Teacher support and guidance in the classroom setting while using the product is a benefit to the overall process from a motivational standpoint, but as noted elsewhere, the product is designed for self-instruction, and as such teacher support is not crucial due to the inclusion of other supports noted in the Task Analysis portion above. Question and answer sessions during the conduct of a class discussion can easily be incorporated into the initial instruction for purposes of feedback, or not, as situations warrant. 4.) The project itself is the most extensive component for instructional purposes. And, finally, 5.) the post-test assessment may then be used to ascertain the effectiveness of the instruction and product.
Evaluation
Key development decisions and justification
Increasing Effectiveness
Coaches - The coaches were used in order to give additional information about the passage or to ask questions. It was our hope that by having the coaches on each page that the students would gain a deeper meaning from the material they were reading. The coaches also provide multiple means of engagement and representation. Glossary- The glossary was created so that the students would have an easy link from the unknown word to its definition, providing multiple means of representation and engagement. Student response prompts- The student response prompts were created as an alternate means of assessment or formative evaluation as well as to provide multiple means of engagement and expression. Multiple links - Multiple links were used in order to increase student interactivity with the book and to provide multiple means of representation and engagement. Pictures-Images were used throughout the Book Builder in order to give students a visual image that corresponded to the content of each page. We hoped that it would also help keep the students engaged while reading the material as well as provide multiple means of representation.
Increasing Ease of Use
Tutorial - A tutorial video was created and incorporated into the project to facilitate the effective use of the product. Table of contents - The table of contents was created in order to allow the student to maneuver throughout the Book Builder. It allows the reader to skip to a certain topic without having to turn page by page. Changed font of glossary links to italics- Changing the glossary links to italics and the multimedia links to bold allowed the reader to differentiate glossary links from multimedia links. Student response areas - Providing a student response area directly in the Book Builder lesson made it easier for students to record their answers. A separate sheet of paper or a separate Word document would be more cumbersome for teachers and students.
Evaluation
Formative Evaluation
We employed the following formative evaluations in our evaluation process: peer evaluations, learner observations, instructional designer observations, and subject matter expert evaluations. We asked our instructional design peers to evaluate our Great Depression Book Builder lesson and provide feedback on their experience and impressions through an online survey. General responses indicated that the site was easy to use, the content was easy to understand, and the lesson had an attractive presentation. Our peers also indicated that they wanted to see more social studies content presented in this fashion. Furthermore, 25% of respondents rated the lesson “very good” and 75% rated it “good”. Components of the lesson that our peers liked best included the audio clips, music, external article, glossary, and student response section. The least liked aspects were the long length of the lesson, overabundance of text, and lack of audio on certain pages. Suggestions for improvement to the lesson indicated a need to reduce the text significantly on many of our pages, perhaps replacing it with a narrator coach or narration clips. Another peer suggested the inclusion of more of what makes the content interactive, meaning more audio, graphics and video links on every page. The utilization of our Coaches could be improved by programming them to suggest clicking on available audio clips. One reviewer indicated this change would make them more appealing and exciting. And finally, there was an appeal for the use of more color on the pages in pursuit of a more attractive design. The three questions that received the lowest rankings were related to learner engagement. This was troubling because increasing learner engagement was a primary goal of the project.
The lesson is interesting and engaging (13% strongly agree, 66% agree)
The lesson has an attractive presentation. (13% strongly agree, 66% agree)
The Book Builder lesson makes me want to learn more about the Great Depression. (25% strongly agree, 63% agree, 13% disagree)
Two members of our design team completed observations of three high school students' interaction with the product interface. Observations were recorded relating to the level of attention, enthusiasm, and reaction to the lesson’s delivery method, content, and student response. Students spent approximately 1 to 5 minutes on each page. Total time to complete the Book Builder ranged from 30 minutes to 1 hour. Generally, students accessed only a few of the interactive audio and video links, stating that they were too long. The students did not use the glossary at all and were inconsistent in launching the coaches. The voice of one of the coaches was difficult to understand, and students found the coaches to be slow and immature. Overall the coaches seemed to be a distraction to the students. Students were also inconsistent in using the student response area. Students asked if they really had to answer all the questions and expressed frustration that the student response area was small and difficult to edit. On some pages, students asked for help in answering the questions. Overall, students seemed to be at least moderately engaged with the lesson although the slow loading speed of the coaches and the pages was a distraction. The content was age- and grade-appropriate. The questions were challenging and required the students to put forth effort and thought. On some occasions, students seemed to make connections between the lesson content and prior learning or personal experiences. One student did not seem to put forth much effort and did not access many links. He attributed this lack of effort to not wanting to wait on the school’s slow internet and said that he would access the links if he were on a faster internet connection.
The subject matter expert (SME) evaluation sought to answer whether or not the content selected was appropriate given the objectives chosen by the team for the Great Depression Book Builder lesson. Overall the SME evaluators found the lesson content to be comprehensive and accurate, indicating that the components of the lessons most useful in helping learners acquire the knowledge and skill were the audio clips as means of demonstrating various music forms of the era; graphics and videos of events, persons, and items to reinforce a better understanding of the challenges and ways of dealing with those challenges during the Great Depression; the integration of student response sections; and the use of coaches to offer explanations or additional insight. Suggestions for improvement related to the reorganization of the hierarchy of the content.
The following changes were made to the Great Depression Book Builder as a result of our formative evaluation:
Added an introductory slide and screencast video explaining the features of Book Builder and our expectations of students (i.e., use the coaches on every page, use the glossary when unfamiliar with a word or term, and access every link)
Changed the voice of the first coach to a female voice which enunciated more clearly
Eliminated text on text-heavy slides
Added interactive components (audio/video links) on slides previously without any links
Changed the names of our coaches to better describe their particular use (Dee Do-You-Know and Relate-It-Ralph)
Sped up page loading by resizing the photos
Distinguished definition links (italicized) from media links (boldfaced)
Reorganized the hierarchy of the material being presented by reordering pagination
Expanded the jazz slide to address the Harlem Renaissance and included it in the glossary.
Deleted the original page 18, Horror Movies part 2, as it was redundant with page 17, Horror Movies part 1
On the Dance Marathon page, added a YouTube clip from the era and deleted the clip to and content about "They Shoot Horses, Don't They?" as a means to keep the project focused on the period rather than later efforts to dramatize the period
Summative Evaluation
We were not able to implement our summative evaluation but did design a comprehensive summative evaluation plan to include a design evaluation, content evaluation, and learner evaluation.
A design evaluation will be conducted to verify that all the planned software design features have been incorporated and are functional. This evaluation will be conducted by a person external to the design team via a checklist. The evaluator will verify Alt Text is available for every photograph; coach graphics and voices are implemented; video, audio, and text links are valid; page layout and text formatting are standard on every page, and color choices are complementary.
A content evaluation will verify that the content (text, audio and video) incorporated in the lesson are accurate, appropriate and all sources are cited correctly. This evaluation will be completed by the SME from The Great Designer’s design team via checklist. The evaluator will verify that the content remains accurate after edits and is appropriately aligned to the defined goal and topic.
The learner evaluation will serve three purposes. 1. Verify that learning has occurred by having students complete online pre-tests and post-tests of the content of the Book Builder lesson. The content questions on both tests will be identical and relate to the content addressed in the Book Builder lesson. 2. Verify that the instructional interface (Book Builder) is understood and easily navigated via direct observation of learner interaction with the Book Builder lesson. Observations will be inputted into the existing Learner Observation Survey. 3. Verify that the instruction was engaging for the learner based on two collection methods: a.) Direct observation of the learner during the summative evaluation event with results inputted into the Learner Observation Survey. b.) Direct questioning of students' engagement and satisfaction with the Great Depression Book Builder via the learner post-test.
The learner evaluation will require both a facilitator and an official observer to be in attendance. A total of five 9th -12th grade student learners will participate in this group evaluation. The students will volunteer based on a request for participants via school announcements. The evaluation requires a technology lab/classroom setting. Each participant will have access to a computer specifically dedicated to this evaluation event. The lab/classroom must be equipped with “fast” access to the Internet. Each computer system will include a mouse, color monitor and working headphones. The lesson will be delivered according to the instructional sequence detailed earlier in this report.
Expected maintenance and distribution requirements
1.) How long will our links to videos remain active? These links may have to be monitored and updated when no longer accessible. 2.) Will the school where this book is going to be used have fast internet access? If the school does not have fast internet access, the book could possibly be used as a Smart Board whole class instruction. 3.) Does the school where the book is used block YouTube videos, and if it does, can exceptions be added to the firewall to allow the specific items being requested to show. 4.) Can archival copies of the videos be downloaded and used for backup purposes to ensure their accessibility in the future under “fair use” or in accordance with Creative Commons attributes?
References Included in Book Builder Project
Batman. (n.d.) In Wikipedia online Retrieved November 11, 12 from http:/en.wikipedia..org/wiki/Batman.
Chackwor. (2012). Gold Diggers of 1933 – "We're In The Money" Retrieved from http:/www.youtube.com/watch?v=UJOjTNuuEVw.
RoboJapan . (2012) Dracula 1931 Trailer. Retrieved from http:/www.youtube.com/watch?v=7Nfmh178L98.
Stead, J. (2008, September 4).Woody Guthrie – Talking Dustbowl Blues. Retrieved from http:/www.youtube.com/watch?v=dkAxuqrVNBNM.
General References
Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The cognitive domain. New York: Longman.
Brown, A. (2011). The essentials of instructional design : connecting fundamental principles with process and practice. Boston: Prentice Hall.
South Carolina Department of Education. (n.d.). End-of-Course Examination Program. SCDE: SCDE. Retrieved September 25, 2012 from the World Wide Web: http://ed.sc.gov/data/eocep/.
Management: Documentation of ISD Process
Group Time Log
The conduct of a synchronous team meeting was not found to be possible due to the varied schedules and obligations of the respective team members. The means of communication throughout the entire project was by means of Discussion Board postings. At the conclusion of the project and at the time of the creation of the final report there were in excess of 460 separate postings in varying categories based on the topic being addressed. If an average of 2 minutes per posting were to be calculated out a total meeting time of 15.3 hours is the amount of time in meeting for the team. For purposes of documentation a screenshot is included below. In addition to Discussion Board postings, the team also met asynchronously via the Blackboard File Exchange, the team __wiki__ (http://thegreatdesigners.wikispaces.com/), email, and Google Docs (the storyboard and final report).
Shawn's Time Log
Date
Time
Task
Notes
10/13/12
1.5 hours
Research
Needs assessment regarding impact of test scores on schools and teachers.
10/14/12
2 hours
Research
Data for needs analysis.
10/15/12
2 hours
Wiki
Write and post needs analysis for wiki.
10/19/12
1 hour
Research
Prepare material for learner analysis.
10/21/12
1 hour
Wiki
Place materials and work on wiki.
11/4/12
1 hour
Wiki
Made adjustments on wiki.
11/6/12
2.5 hours
Content
Edited content captions, etc. published product and added link to wiki.
11/9/12
1 hour
Content
Researched and added content.
11/11/12
2 hours
Wiki
Added and worked with Bookbuilder vocabulary and arrangement.
11/24/12
3 hours
Content
Made revisions to project including suggestions for revision and addition of additional content.
11/25/12
3 hours
Wiki/content
Made additional revisions to project, worked on screencast, made revisions to wiki and completed SME evaluation.
12/1/12
2 hours
Final Report
Made edits and additions to final report.
12/2/12
2 hours
Final Report
Made edits and additions to final report.
Total
24 hours
Susan's Time Log
Date
Time
Task
Notes
10/9/12
2 hours
Graphics
Logo Search/creation
10/10/12
1 hour
Graphics
Logo modifications/color scheme
10/13/12
1 hour
Research
Research GD topic -intro
10/15/12
1 hour
Research
Researched/Refined and inserted learning obj.to align with assessment
10/17/12
1 hour
Research
Researched goal/obj. attached rough copy to file exchange
10/18/12
30 min
Development
Worked on flow chart
10/19/12
1 hour
Development
Worked on flow chart
10/25/12
30min
Research
Researched graphic design-posted notes
10/27/12
1 hour
Content
Posted graphics and text to Google Docs.
10/30/12
1 hour
Compose
Posted graphics &text to Google Docs../added graphic info
11/02/12
1 1/2 hr.
Graphics
Obtained photos and content for Google Docs./edited
11/03/12
2 hr.
Compose
Researched and inserted info into BookBuilder
11/05/12
1 hr.
Compose
Added content titles/questions/edited Book Builder
11/06/12
1 hr.
Compose
Researching/adding content & pics. to glossary
11/11/12
30 min.
Compose
Input survey questions into online survey
11/16/12
1 1/2
Observation
Conduct and Input observation data for two high school students
11/20/12
1 hr.
Compose
Created Screen Cast Video
11/24/1
30 min
Compose
Created Pretest
11/25/12
30 min
Revision
Reworked Pretest and Posttest
11/28/12
2 hrs.
Compose
Final report -writing summaries/editing/creation of diagram
11/29/12
2 hrs.
Compose
Final report
12/01/12
1 hr.
Revise
Final report
Total
23.5 hours
Mary Ann's Time Log
Date
Time
Task
Notes
10/08/12
1hr.
Content Search
The Great Depression
10/09/12
1 hr.
Content Search
Music -Great Depression, Family Life/Youtube Videos
10/10/12
30 min.
Content Search
UDL - Great Depression
10/12/12
1hr..
Content Search
PowerPoint Presentations - Inserted Content Files - Wiki Page
10/14/12
1hr.
Content Search
Family Life in SC - Great Depression
10/15/12
1 hr.
Final Project Page
Content Search - Reference Page
10/21/12
2 hrs
Content Research
Composed Analysis Page
10/25/12
1.5 hrs.
Documents
Storyboard Content
10/28/12
1 hr.
Content Development Research
PowerPoint Slides/Storyboard
10/29/12
2 hrs.
Content Development Research
Top 7- Design Placed on Wiki
11/09/12
2 hrs.
References
Composed Reference Page
11/10/12
2 hrs.
Evaluation
Composed Survey Questions - Posted to File Exchange
11/11/12
2. hrs.
References
Posted to Wiki - File Exchange
11/21/12
2 hrs.
Final Report
Worked on report/ google docs.
11/22/12
2 hrs.
Final Report
Worked on report/ google docs.
Total
22 hrs.
Barbara's Time Log
Date
Time
Task
Notes
10/8/2012
1.0 hr
Research
Worked within Bookbuilder site to become familiar with tools and possible object types.
10/9/2012
2.0 hr
Research
Read GD content by sub-areas posted on our wiki, researched Hooverville, Dust Bowl and those less affected by GD on www.
10/11/2012
.30 min
Research
Searched within The Commons for photos suitable for the group wiki and Bookbuilder end product, non-copyrighted photos.
10/14/2012
1.5 hr
Res/compose
Wrote and submitted short intro for wiki, explored types of pre-post testing models for the evaluation of software.
10/19/12
2.0 hr
research
worked on concepts of needs, task, learner and content analysis. Read
10/20/12
2.5 hr
compose
worked on the composition of task analysis and to focus the project topic, worked on ideas for formative evaluations, worked on flow chart for task analysis, evaluated which tool to use for the flow chart.
10/27/12
1.5
research
photographs
10/28/12
2.5
compose
text on movies and coaches and evaluation methods
10/29/12
1.5
compose
uploaded graphics and text
10/30/12
1.0
research
newspapers and magazines topic content
11/2/12
2.0
Edit wiki
Added content and edited txt
11/4/12
2.0
compose
added and organized 'coach' content and tested links of bookbuilder proj.
11/5/12
1.0
edit
Edited book builder content
11/7/12
2.0
research
Researched evaluation measures appropriate for our project.
11/8/12
1.5
compose
Composed the outline and basics of evaluation plan
11/9/12
2.0
compose
Further defined the eval plan and methodology for implementation
11/11/12
1.0
testing
Tested the surveys and Bb book
11/17/12
.30
Research
Summative Evaluation Methods in book
11/18/12
1.5
administered
Administered a student observation survey for Bookbuilder lesson, entered info online.
11/19/12
2.0
Composed
Compiled, evaluated, summarized and posted Peer evaluation results to the wiki site.
11/21/12
1.5
Composed
Composed summative evaluation plan.
11/23/12
1.0
Revisions
Revised summative evaluation content and uploaded draft.
11/25/12
1.5
composed
Worked on the composition of the screencast video content related to evaluation planning. Edited Wiki.
11/26/12
1.5
composed
Created screencast video and embedded into wiki
12/01/12
1.0
reviews/edits
Final project
12/02/12
1.0
reviews/edit
Final Project
Total
39.0
Erin's Time Log
Date
Time
Task
Notes
10/09/2012
0.5 hours
Wiki set-up
Sent invitations to all team members Developed pages for wiki
10/12/2012
3 hours
Wiki set-up
Inserted team and individual time logs Ordered pages Researched color scheme Posted on discussion board about tasks to-do
10/15/2012
2.25 hours
Needs Assessment
Proofreading, editing, and consolidating team members' contributions
10/21/2012
4 hours
Needs Analysis Graphic Organizer
Completed the Needs Analysis Graphic Organizer and gave feedback on Bb discussion board re: other analyses
10/27/2012
4 hours
Storyboard
Set up and shared storyboard template on Google Docs, communicated with team re: storyboard template on Bb discussion board, started threads on storyboard development and coaches
11/01/2012
2.5 hours
Storyboard
Content for storyboard, edited and revised
11/03/2012
4 hours
Book Builder
Added content to storyboard, developed first draft of Book Builder from storyboard, communicated progress with Dr. Wissick and team
11/05/2012
1.5 hours
Wiki
Edited Design content, drafted Development content, embedded anchors and links on Analysis page, embedded table of contents on Home page, Design page, and Time Log page
11/06/2012
1 hour
Bookbuilder
Edited, completed student response areas, inserted link on wiki
11/12/2012
5 hours
Evaluation
Edited, revised, and added to formative evaluation survey; created survey on Google Docs; added content to Implementation and Evaluation pages on wiki; created observation form on Google Docs; edited Book Builder; emailed classmates links to Book Builder, evaluation, and wiki
11/13/2012
1 hour
Communication
Set up Week 13 group discussion board posts identifying tasks to be accomplished and and decisions to be made.
11/18/2012
2 hours
Evaluation & wiki
Summarized observation evaluation and posted on wiki, cleaned up and edited wiki
11/25/2012
5 hours
Screencast, wiki, communication
Created and embedded screencast video, edited wiki, communicated with group members on email and Bb discussion board
11/26/2012
1 hour
Screencast, wiki, communication
Resized screencast video; edit, revised, and reorganized wiki, communicated with group members on email and Bb discussion board. Fixed anchors and links on implementation page.
11/27/2012
1 hour
Screencast, wiki
Tried to assist team member in embedding video. Unable to embed. Posted link instead. Fixed anchors and links on Time Log page.
11/29/2012
6 hours
Final report
Drafted introduction, analysis, and design & development sections
Executive Summary
The Great Designers developed a Book Builder (www.bookbuilder.cast.org/) product for high school students which explores the socio-economic impact of the Great Depression,specifically how people relaxed and entertained themselves during these harsh times. Our creative instructional design approach to this project incorporated text, video, photos, and music clips to create a highly interactive and engaging learning experience. Our goal was to create a multimedia product that while specific in its scope (i.e., the impact of the Great Depression on leisure and entertainment), would be engaging and stimulate student interest by appealing to students' need for more interactive content. Universal Design for Learning (UDL) materials were used to provide alternative pathways to success including multiple modes of presentation, varied levels of support and challenge, and options for recruiting and sustaining interest and motivation. Our product was designed to be self-instructional and can be used either in the classroom or as homework. We incorporated the ADDIE framework in our instructional design process. During the analysis stage we completed the following analyses: needs analysis, goal analysis, learner analysis, task analysis, and content analysis. During the formative evaluation process, the lesson was administered to students, peer evaluators, and subject matter experts. Modifications were completed to make the lesson more accessible and engaging to students. The summative evaluation measures include measures of students’ learning as well as measures of student engagement.Analysis
Objectives
The purpose of the lesson was to raise student scores on the End of Course (EOC) exam in required South Carolina high school U.S. History and Constitution courses by raising student interest and engagement. The Great Designers adopted state curriculum standards for U.S. History and Constitution as the project objectives. The specific standards addressed by the lesson are listed below with the relevant text regarding lesson objectives shown in bold:- USHC-6.1 Explain the impact of the changes in the 1920s on the economy, society, and culture, including the expansion of mass production techniques, the invention of new home appliances, the introduction of the installment plan, the role of transportation in changing urban life, the effect of radio and movies in creating a national mass culture, and the cultural changes exemplified by the Harlem Renaissance.
- USHC-6.3 Explain the causes and consequencesof the Great Depression, including the disparities in income and wealth distribution; the collapse of the farm economy and the effects of the Dust Bowl; limited governmental regulation; taxes, investment; and stock market speculation; policies of the federal government and the Federal Reserve System; and the effects of the Depression on the people.
The ultimate goal of our lesson was to generate an increase in student engagement, and thereby an increase in student achievement, on U.S. History and Constitution End of Course (EOC) exam scores. The incorporation of technology in the presentation of material was an over-riding consideration in keeping with the desire to incorporate principles of UDL for effectiveness and accessibility of content information.Analysis Process
The Great Designers completed a needs analysis, goal analysis, learner analysis, task analysis, and content analysis. Each team member led one analysis with contributions from the other team members. To summarize the results of our analyses, each team member developed a graphic organizer to visually present his or her assigned analysis. A needs analysis was essential to ensure that there was in fact a need for the lesson and to determine the specific focus of the need (i.e., identifying relevant subtopics within the broad topic of the Great Depression). After determining the need, The Great Designers conducted a goal analysis based on the curriculum standards and goals identified in the needs analysis. A learner analysis was essential since most of the team members did not have experience working with the high school population or teaching U.S. History and Constitution. A task analysis identified the general format of and activities to be included in the Book Builder lesson, an important step since the purpose of the lesson was to create a more interactive and engaging learning activity for students. Despite narrowing the broad topic of the Great Depression down to the subtopic of leisure and entertainment, this subtopic was still too broad for the current Book Builder project. A content analysis enabled the team to further narrow the focus to specific types of leisure and entertainment activities and ensure content veracity and usefulness.Needs Analysis
The initial analysis for topic selection was largely derived from examination of information made available from the South Carolina Department of Education regarding student performance on End of Course (EOC) assessment data (SCDE EOCEP, 2012). U.S. History and Constitution is a required course for high school graduation in the state of South Carolina. In 2005 the SC Department of Education implemented an EOC exam for all students enrolled in U.S. History and Constitution. Since the implementation of the EOC exam, scores have been consistently low with over 71% of students earning a D or F. Additionally, no more than 3% of students have attained a grade of A in the most recent report available in 2011. The EOC exam not only comprises 20% of students' final grades, but also impacts schools' and districts' report cards and state funding. Anecdotal evidence suggests that the low EOC exam performance is due to a mismatch between student needs and teacher instructional delivery methods as well as low student interest in U.S. history in general. Furthermore, students do not see the importance of connecting their historical past to their present and future. The Great Depression is included in the curriculum standards for the U.S. History course and consequently is included on the EOC exam. This evidence supports the general need to retool teaching strategies and learning opportunities in U.S. History and Constitution classes and the era of the Great Depression was chosen due to a great amount of resources that can be successfully incorporated into a product such as Book Builder. Goal AnalysisAs stated in our initial Goal Analysis on the project wiki, our goal was to, “generate student interest in eleventh grade United States History.” With this in mind objectives were selected that were both supported by and found within S.C. state standards for U.S. History and Constitution as noted above. These objectives were selected to address the cognitive domains found in Bloom’s Taxonomy regarding the lower levels of cognition (identifying), and expanding somewhat to incorporate this information into higher levels for purposes of explanation (Bloom, Engelhart, Furst, Hill, Krathwohl, 1956, p. 95, 2011). Upon completion of the instructional unit measurements may then be made by means of data collected from the post-test to determine the effectiveness of the instructional module, and if necessary, the incorporation of further instruction.
Learner Analysis
The student traditionally enrolled in U.S. History and Constitution is 16 to 17 years old, in the 11th grade, male or female, and from diverse racial and ethnic groups. The ability level of students range from challenged to gifted and talented, but the greater majority fall in the average range, and this is the case with technical proficiency as well. Under normal circumstances an intervention of this type might be targeted towards the lower performers, but given the data that over 70% of all test-takers statewide are at a below average or failing level it was reasonable to assert that the average student still falls within the scope of students requiring an intervention such as The Great Designer’s project. Likewise, the technical ability of students may be affected by the socio-economic conditions found in the school utilizing the intervention which may limit extended knowledge of the integrated technologies within the project. This may also pose limitations insofar as that students utilizing the project may have limitations with access to the internet at home which may, in turn, limit usage for purposes of reinforcement and/or homework beyond the classroom setting.Task Analysis
The product was created and developed to be self-instructional in nature, and with this in mind efforts were made to make it as user-friendly as possible. Tasks incorporated into the product itself were largely universal features found in traditional web pages such as hyperlinks to supporting materials, videos, audio segments, etc. Where the incorporated tasks were less intuitive, i.e. use of the glossary feature, etc., an explanatory video was included at the outset to explain those features more in-depth so that the user would be able to successfully navigate the product with little difficulty. Aiding these modification decisions were data from formative evaluations which enabled the team to make decisions regarding the content presentation, page formatting, and user enabled features.Content Analysis
Of all of our analyses, the content analysis probably changed the most throughout the instructional design process. Because the Great Depression is a very broad topic, we found it necessary to narrow the lesson topic and content several times during the design, development, implementation, and evaluation phases. As can be seen in our content analysis presented on our wiki, our original scope of content for inclusion in the project was much broader, including causes of the Great Depression, the New Deal, and general vocabulary like shanty towns, inflation, and recession. After much discussion and trial-and-error, we gradually refined the topic to focus specifically on leisure and entertainment (see the Music/Arts & Culture section of the content analysis). We ensured our content was valid and complete by aligning the content with the aforementioned curriculum standards as well as by conducting formative subject matter expert (SME) evaluations. Based on data from our SME evaluators and formative evaluation surveys, we modified our content by once again narrowing it to directly support the objectives chosen in the goal and content analysis.Design and Development
Description of the instruction, setting, activities, and sequence
The Book Builder lesson is designed to be implemented in a high school U.S. History and Constitution classroom with access to computers and high-speed internet (see below for details). The following sequence and explanation for instruction is recommended:Development process supporting the instructional approach
Major components
The major components for instruction in our project begin with 1.) a pretest assessment of current student ability, and possibly to activate prior knowledge with respect to the content. 2.) The introductory presentation will then be followed by the use of the product for support and further enquiry. 3.) Teacher support and guidance in the classroom setting while using the product is a benefit to the overall process from a motivational standpoint, but as noted elsewhere, the product is designed for self-instruction, and as such teacher support is not crucial due to the inclusion of other supports noted in the Task Analysis portion above. Question and answer sessions during the conduct of a class discussion can easily be incorporated into the initial instruction for purposes of feedback, or not, as situations warrant. 4.) The project itself is the most extensive component for instructional purposes. And, finally, 5.) the post-test assessment may then be used to ascertain the effectiveness of the instruction and product.Evaluation
Key development decisions and justification
Increasing Effectiveness
Coaches - The coaches were used in order to give additional information about the passage or to ask questions. It was our hope that by having the coaches on each page that the students would gain a deeper meaning from the material they were reading. The coaches also provide multiple means of engagement and representation.Glossary- The glossary was created so that the students would have an easy link from the unknown word to its definition, providing multiple means of representation and engagement.
Student response prompts- The student response prompts were created as an alternate means of assessment or formative evaluation as well as to provide multiple means of engagement and expression.
Multiple links - Multiple links were used in order to increase student interactivity with the book and to provide multiple means of representation and engagement.
Pictures-Images were used throughout the Book Builder in order to give students a visual image that corresponded to the content of each page. We hoped that it would also help keep the students engaged while reading the material as well as provide multiple means of representation.
Increasing Ease of Use
Tutorial - A tutorial video was created and incorporated into the project to facilitate the effective use of the product.Table of contents - The table of contents was created in order to allow the student to maneuver throughout the Book Builder. It allows the reader to skip to a certain topic without having to turn page by page.
Changed font of glossary links to italics- Changing the glossary links to italics and the multimedia links to bold allowed the reader to differentiate glossary links from multimedia links.
Student response areas - Providing a student response area directly in the Book Builder lesson made it easier for students to record their answers. A separate sheet of paper or a separate Word document would be more cumbersome for teachers and students.
Evaluation
Formative Evaluation
We employed the following formative evaluations in our evaluation process: peer evaluations, learner observations, instructional designer observations, and subject matter expert evaluations.We asked our instructional design peers to evaluate our Great Depression Book Builder lesson and provide feedback on their experience and impressions through an online survey. General responses indicated that the site was easy to use, the content was easy to understand, and the lesson had an attractive presentation. Our peers also indicated that they wanted to see more social studies content presented in this fashion. Furthermore, 25% of respondents rated the lesson “very good” and 75% rated it “good”. Components of the lesson that our peers liked best included the audio clips, music, external article, glossary, and student response section. The least liked aspects were the long length of the lesson, overabundance of text, and lack of audio on certain pages. Suggestions for improvement to the lesson indicated a need to reduce the text significantly on many of our pages, perhaps replacing it with a narrator coach or narration clips. Another peer suggested the inclusion of more of what makes the content interactive, meaning more audio, graphics and video links on every page. The utilization of our Coaches could be improved by programming them to suggest clicking on available audio clips. One reviewer indicated this change would make them more appealing and exciting. And finally, there was an appeal for the use of more color on the pages in pursuit of a more attractive design. The three questions that received the lowest rankings were related to learner engagement. This was troubling because increasing learner engagement was a primary goal of the project.
Two members of our design team completed observations of three high school students' interaction with the product interface. Observations were recorded relating to the level of attention, enthusiasm, and reaction to the lesson’s delivery method, content, and student response. Students spent approximately 1 to 5 minutes on each page. Total time to complete the Book Builder ranged from 30 minutes to 1 hour. Generally, students accessed only a few of the interactive audio and video links, stating that they were too long. The students did not use the glossary at all and were inconsistent in launching the coaches. The voice of one of the coaches was difficult to understand, and students found the coaches to be slow and immature. Overall the coaches seemed to be a distraction to the students. Students were also inconsistent in using the student response area. Students asked if they really had to answer all the questions and expressed frustration that the student response area was small and difficult to edit. On some pages, students asked for help in answering the questions. Overall, students seemed to be at least moderately engaged with the lesson although the slow loading speed of the coaches and the pages was a distraction. The content was age- and grade-appropriate. The questions were challenging and required the students to put forth effort and thought. On some occasions, students seemed to make connections between the lesson content and prior learning or personal experiences. One student did not seem to put forth much effort and did not access many links. He attributed this lack of effort to not wanting to wait on the school’s slow internet and said that he would access the links if he were on a faster internet connection.
The subject matter expert (SME) evaluation sought to answer whether or not the content selected was appropriate given the objectives chosen by the team for the Great Depression Book Builder lesson. Overall the SME evaluators found the lesson content to be comprehensive and accurate, indicating that the components of the lessons most useful in helping learners acquire the knowledge and skill were the audio clips as means of demonstrating various music forms of the era; graphics and videos of events, persons, and items to reinforce a better understanding of the challenges and ways of dealing with those challenges during the Great Depression; the integration of student response sections; and the use of coaches to offer explanations or additional insight. Suggestions for improvement related to the reorganization of the hierarchy of the content.
The following changes were made to the Great Depression Book Builder as a result of our formative evaluation:
Summative Evaluation
We were not able to implement our summative evaluation but did design a comprehensive summative evaluation plan to include a design evaluation, content evaluation, and learner evaluation.A design evaluation will be conducted to verify that all the planned software design features have been incorporated and are functional. This evaluation will be conducted by a person external to the design team via a checklist. The evaluator will verify Alt Text is available for every photograph; coach graphics and voices are implemented; video, audio, and text links are valid; page layout and text formatting are standard on every page, and color choices are complementary.
A content evaluation will verify that the content (text, audio and video) incorporated in the lesson are accurate, appropriate and all sources are cited correctly. This evaluation will be completed by the SME from The Great Designer’s design team via checklist. The evaluator will verify that the content remains accurate after edits and is appropriately aligned to the defined goal and topic.
The learner evaluation will serve three purposes.
1. Verify that learning has occurred by having students complete online pre-tests and post-tests of the content of the Book Builder lesson. The content questions on both tests will be identical and relate to the content addressed in the Book Builder lesson.
2. Verify that the instructional interface (Book Builder) is understood and easily navigated via direct observation of learner interaction with the Book Builder lesson. Observations will be inputted into the existing Learner Observation Survey.
3. Verify that the instruction was engaging for the learner based on two collection methods:
a.) Direct observation of the learner during the summative evaluation event with results inputted into the Learner Observation Survey.
b.) Direct questioning of students' engagement and satisfaction with the Great Depression Book Builder via the learner post-test.
The learner evaluation will require both a facilitator and an official observer to be in attendance. A total of five 9th -12th grade student learners will participate in this group evaluation. The students will volunteer based on a request for participants via school announcements. The evaluation requires a technology lab/classroom setting. Each participant will have access to a computer specifically dedicated to this evaluation event. The lab/classroom must be equipped with “fast” access to the Internet. Each computer system will include a mouse, color monitor and working headphones. The lesson will be delivered according to the instructional sequence detailed earlier in this report.
Expected maintenance and distribution requirements
1.) How long will our links to videos remain active? These links may have to be monitored and updated when no longer accessible.2.) Will the school where this book is going to be used have fast internet access? If the school does not have fast internet access, the book could possibly be used as a Smart Board whole class instruction.
3.) Does the school where the book is used block YouTube videos, and if it does, can exceptions be added to the firewall to allow the specific items being requested to show.
4.) Can archival copies of the videos be downloaded and used for backup purposes to ensure their accessibility in the future under “fair use” or in accordance with Creative Commons attributes?
References Included in Book Builder Project
Batman. (n.d.) In Wikipedia online Retrieved November 11, 12 fromhttp:/en.wikipedia..org/wiki/Batman.
Chackwor. (2012). Gold Diggers of 1933 – "We're In The Money" Retrieved from
http:/www.youtube.com/watch?v=UJOjTNuuEVw.
Ganzel, B, & Reinhardt, C. (2003). Having Fun-Family Time. Retrieved September 11, 2012 from
http://www.livinghistoryfarm.org/farminginthe30s/life_20.html.
Great Depression. (n.d.) In Wikipedia online. Retrieved November 11, 2012 from
http://en.wikipedia.org/wiki/Great_Depression.
Hindenburg Disaster. (n. d.). In Wikipedia online. Retrieved November 11, 2012 from
http://en.wikipedia.org/wiki/Hindenburg_disaster.
NPR. 2011. A Depression Era Anthem For Our Times. Retrieved from
http://www.npr.org/2008/11/15/96654742/a-depression-era-anthem-for-our- times.
NPR, 2010. Brother Can You Spare A Dime. Retrieved from
http://pd.npr.org/anon.nprmp3/npr/specialmusic/2009/10/20091022_specialmusicbrother.mp3
RoboJapan . (2012) Dracula 1931 Trailer. Retrieved from
http:/www.youtube.com/watch?v=7Nfmh178L98.
Stead, J. (2008, September 4).Woody Guthrie – Talking Dustbowl Blues. Retrieved from
http:/www.youtube.com/watch?v=dkAxuqrVNBNM.
General References
Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The cognitive domain. New York: Longman.Brown, A. (2011). The essentials of instructional design : connecting fundamental principles with
process and practice. Boston: Prentice Hall.
South Carolina Department of Education. (n.d.). End-of-Course Examination Program. SCDE: SCDE.
Retrieved September 25, 2012 from the World Wide Web: http://ed.sc.gov/data/eocep/.
Management: Documentation of ISD Process
Group Time Log
The conduct of a synchronous team meeting was not found to be possible due to the varied schedules and obligations of the respective team members. The means of communication throughout the entire project was by means of Discussion Board postings. At the conclusion of the project and at the time of the creation of the final report there were in excess of 460 separate postings in varying categories based on the topic being addressed. If an average of 2 minutes per posting were to be calculated out a total meeting time of 15.3 hours is the amount of time in meeting for the team. For purposes of documentation a screenshot is included below. In addition to Discussion Board postings, the team also met asynchronously via the Blackboard File Exchange, the team __wiki__ (http://thegreatdesigners.wikispaces.com/), email, and Google Docs (the storyboard and final report).Shawn's Time Log
Susan's Time Log
Mary Ann's Time Log
Barbara's Time Log
flow chart for task analysis, evaluated which tool to use for the flow chart.
Erin's Time Log
Developed pages for wiki
Ordered pages
Researched color scheme
Posted on discussion board about tasks to-do
Graphic Organizer
on Bb discussion board re: other analyses
with team re: storyboard template on Bb discussion board, started threads
on storyboard development and coaches
communicated progress with Dr. Wissick and team
Analysis page, embedded table of contents on Home page, Design page, and Time Log page
added content to Implementation and Evaluation pages on wiki; created observation form on
Google Docs; edited Book Builder; emailed classmates links to Book Builder, evaluation, and wiki
and decisions to be made.
communication
email and Bb discussion board
communication
on email and Bb discussion board. Fixed anchors and links on implementation page.
and links on Time Log page.
Team Totals